|
|
The algorithm for the detection of positive and negative text
Musil, David ; Harár, Pavol (referee) ; Povoda, Lukáš (advisor)
As information and communication technology develops swiftly, amount of information produced by various sources grows as well. Sorting and obtaining knowledge from this data requires significant effort which is not ensured easily by a human, meaning machine processing is taking place. Acquiring emotion from text data is an interesting area of research and it’s going through considerable expansion while being used widely. Purpose of this thesis is to create a system for positive and negative emotion detection from text along with evaluation of its performance. System was created with Java programming language and it allows training with use of large amount of data (known as Big Data), exploiting Spark library. Thesis describes structure and handling text from database used as source of input data. Classificator model was created with use of Support Vector Machines and optimized by the n-grams method.
|
|
|
Text document plagiarism detector
Kořínek, Lukáš ; Horák, Karel (referee) ; Petyovský, Petr (advisor)
This diploma thesis is concerned with research on available methods of plagiarism detection and then with design and implementation of such detector. Primary aim is to detect plagiarism within academic works or theses issued at BUT. The detector uses sophisticated preprocessing algorithms to store documents in its own corpus (document database). Implemented comparison algorithms are designed for parallel execution on graphical processing units and they compare a single subject document against all other documents within the corpus in the shortest time possible, enabling near real-time detection while maintaining acceptable quality of output.
|
|
|
N-grams in the speech of Czech and native speakers of English
Zvěřinová, Simona ; Gráf, Tomáš (advisor) ; Tichý, Ondřej (referee)
The diploma thesis is concerned with the analysis of recurrent word-combinations in the speech of advanced Czech speakers of English and native speakers of English. The data used for the analysis is extracted from two corpora, learner corpus LINDSEI and native speaker corpus LOCNEC. The aim of the thesis is to compare the two groups of speakers, determine differences in their use of recurrent word-combinations and compare the findings to previous studies involving speakers of different languages. The quantitative analysis is performed on a sample of 50 speakers from each corpus and the frequency data is used to compare the two groups as to the number of types of word-combinations they use and how frequently they do so. The qualitative analysis is performed on a sample of 15 speakers from each corpus to determine functional differences. Four categories of word-combinations are determined in the analysis. In the conclusion, the quantitative and qualitative findings are compared to previous research involving speakers of different languages. Keywords: spoken language, learner language, n-grams, n-gram analysis, recurrent word- combinations, lexical bundles, learner corpus
|
|
|
Text document plagiarism detector
Kořínek, Lukáš ; Horák, Karel (referee) ; Petyovský, Petr (advisor)
This diploma thesis is concerned with research on available methods of plagiarism detection and then with design and implementation of such detector. Primary aim is to detect plagiarism within academic works or theses issued at BUT. The detector uses sophisticated preprocessing algorithms to store documents in its own corpus (document database). Implemented comparison algorithms are designed for parallel execution on graphical processing units and they compare a single subject document against all other documents within the corpus in the shortest time possible, enabling near real-time detection while maintaining acceptable quality of output.
|
|
|
N-grams in the speech of Czech and native speakers of English
Zvěřinová, Simona ; Gráf, Tomáš (advisor) ; Tichý, Ondřej (referee)
The diploma thesis is concerned with the analysis of recurrent word-combinations in the speech of advanced Czech speakers of English and native speakers of English. The data used for the analysis is extracted from two corpora, learner corpus LINDSEI and native speaker corpus LOCNEC. The aim of the thesis is to compare the two groups of speakers, determine differences in their use of recurrent word-combinations and compare the findings to previous studies involving speakers of different languages. The quantitative analysis is performed on a sample of 50 speakers from each corpus and the frequency data is used to compare the two groups as to the number of types of word-combinations they use and how frequently they do so. The qualitative analysis is performed on a sample of 15 speakers from each corpus to determine functional differences. Four categories of word-combinations are determined in the analysis. In the conclusion, the quantitative and qualitative findings are compared to previous research involving speakers of different languages. Keywords: spoken language, learner language, n-grams, n-gram analysis, recurrent word- combinations, lexical bundles, learner corpus
|
|
|
Framework for information extraction from the large language data sets
Kuboň, David ; Križ, Vincent (advisor) ; Bednárek, David (referee)
This thesis describes the FAFEFI program that focuses on n-gram and skip-gram extraction from large data sets. The thesis presents two different approaches to passing input data to the program. It also describes the design of data structures for n-gram and skip-gram representation within computer memory, the algorithm of n-gram and skip-gram extraction, memory-friendly options of saving extracted data and their final composition into output feature vectors. It also offers a variety of extra functions such as line filter and line modifier and a great deal of configurable parameters ranging from in-file separators to formatting the names of output files. Moreover, the program provides a differentiation in its activity by enabling saving data just after extraction from the train set and brings tools for cluster parallelization. Powered by TCPDF (www.tcpdf.org)
|
|
|
The algorithm for the detection of positive and negative text
Musil, David ; Harár, Pavol (referee) ; Povoda, Lukáš (advisor)
As information and communication technology develops swiftly, amount of information produced by various sources grows as well. Sorting and obtaining knowledge from this data requires significant effort which is not ensured easily by a human, meaning machine processing is taking place. Acquiring emotion from text data is an interesting area of research and it’s going through considerable expansion while being used widely. Purpose of this thesis is to create a system for positive and negative emotion detection from text along with evaluation of its performance. System was created with Java programming language and it allows training with use of large amount of data (known as Big Data), exploiting Spark library. Thesis describes structure and handling text from database used as source of input data. Classificator model was created with use of Support Vector Machines and optimized by the n-grams method.
|
|
|
Representation of Text and Its Influence on Categorization
Šabatka, Ondřej ; Chmelař, Petr (referee) ; Bartík, Vladimír (advisor)
The thesis deals with machine processing of textual data. In the theoretical part, issues related to natural language processing are described and different ways of pre-processing and representation of text are also introduced. The thesis also focuses on the usage of N-grams as features for document representation and describes some algorithms used for their extraction. The next part includes an outline of classification methods used. In the practical part, an application for pre-processing and creation of different textual data representations is suggested and implemented. Within the experiments made, the influence of these representations on accuracy of classification algorithms is analysed.
|
|
|
Effective methods of plagiarism detectios in large document repositories
Přibil, Jiří ; Jiroušek, Radim (advisor) ; Strossa, Petr (referee) ; Snášel, Václav (referee)
The work focuses on issues of plagiarism detection in large document repositories. Taking into account real situation that needs to be addressed now in the university environment in the Czech Republic and proposes a system that will be able to carry out this analysis in real time and yet be able to capture the widest possible range of plagiarism methods. The main contribution of this work is taking the definition of so-called unordered n-grams - {n}-grams - which can be used just to detect some forms of advanced plagiarism methods. All cited recommendations that relate to the various components of the system to detect plagiarism - preprocessing the document before document insertion into the corpus, the representation of documents in document storage, identification of potential sources of plagiarism to calculate rates of similarity; visualization analysis of plagiarism - are subject to discussion and appropriately quantified. The result is a set of design parameters of the system so that it can in detect plagiarism in the Czech language language quickly, accurately and yet in most forms.
|
guest ::
